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1.  INTRODUCTION:  The  proposed  research  is  focused  on  restoration  of  hand  motor  and  sensory 
functions  by  utilizing  a  direct  electrical  interface  with  residual  peripheral  nerves.  The  direct 
connection  with  the  residual  nerves  will  enable  the  patient  to  have  intuitive  control  over  and 
receive  touch  sensation  from  a  prosthetic  hand  that  are  not  provided  by  current  forearm 
prostheses.  The  improvement  in  intuitive  control  and  the  providing  of  sensory  feedback  will 
allow  patients  to  use  highly  articulate  prosthetic  hands  with  improved  long-term  functional 
outcomes  for  military  personnel  and  civilians  with  a  forearm  amputation.  Based  on  preliminary 
studies  in  peripheral  nerves  it  is  possible  to  decode  finger  movements  from  electrophysiological 
signals  recorded  from  peripheral  nerves,  and  to  evoke  somatosensory  perceptions  through 
micro-stimulation  of  peripheral  nerves.  The  proposed  research  will  determine  the  type  and 
complexity  of  movements  that  can  be  controlled  by  a  direction  connection  to  arm  nerves,  and 
will  also  determine  the  type  and  range  touch  sensations  that  can  be  provide  through  a  direct 
connection  to  residual  nerves. 

2.  KEYWORDS:  Peripheral  Nerve  Interface,  Prosthetic  Hand,  Neural  Prosthesis,  Sensory  Feedback, 
Micro-stimulation,  Electrophysiology,  Action  Potentials,  Micro-electrode,  poly-Longitudinal 
Intrafascicular  Electrode  (poly-LIFE) 

3.  ACCOMPLISHMENTS:  Provided  in  italics  in  appropriate  sections. 

What  were  the  major  goals  of  the  project? 

Specific  Aim  1:  Dexterous  control  of,  and  sensory  feedback  from,  an  advanced  prosthetic 
hand  will  be  provided  using  micro-electrode  arrays  implanted  in  residual  peripheral  nerves 

Major  Goal  1:  Preparation  for  Studies 

Subtask  1.1:  Regulatory  Approvals  (Months  1-4) 

•  UU:  Dr.  Hutchinson,  Nousheen  Alasti;  oversee  writing  and  submission  of  protocol 
and  associated  documents  for  IRB  and  HRPO. 

Will  no  longer  be  collaborating  with  University  of  Utah  and  will  be  formally 
requesting  cancelation  of  University  of  Utah  subcontract. 

•  Mayo:  Dr.  Smith  and  Mayo  Study  Coordinator  oversee  writing  and  submission  of 
protocol  and  associated  documents  for  IRB  and  HRPO. 

•  Dr.  Greger;  assist  in  writing  and  submission  of  IRB  and  HRPO  documents. 

80%  complete.  Draft  of  IRB  protocol  written  and  will  be  submitted  to  Mayo  Clinic  IRB 
and  HRPO  by  30-November-2015. 

Subtask  1.2:  Micro-electrode  Arrays.  (Months  4  -  30) 

•  Production  of  9  human-ready  Sputtered  Iridium-Oxide  (SIROF)  micro-electrode 
arrays  consisting  of  100  electrodes. 

25%  complete.  Design  and  drawings  of  micro-electrode  arrays  complete. 
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•  Measure  impedances  on  all  electrodes  in  each  array  prior  to  sterilization,  and  if 
possible  after  explantation  at  the  end  of  the  study. 

50%  complete.  Instrumentation  to  obtain  impedances  on  micro-electrode  arrays  at 
multiple  frequencies  currently  in  place  and  validated. 

•  Obtain  light  microscope  images  of  the  arrays  prior  to  implantation  and  after 
explantation. 

30%  complete.  Contract  in  place  to  perform  light  and  electron  microscopy  on  the 
micro-electrode  arrays  with  the  Aberration  Corrected  Electron  Microscopy  core 
facility  at  ASU.  Have  obtained  test  images. 


Major  Task  2:  Implantation  and  explantation  of  micro-electrode  arrays  in  a  residual  nerve 
of  patients  with  trans-humeral,  trans-radial,  or  elbow  disarticulation  amputations 

Subtask  2.1:  Patient  Recruitment  (Months  4  -  30) 

•  Volunteers  will  be  recruited  and  informed  consent  obtained  at  the  Mayo  Clinic  using 
the  procedure  in  the  approved  IRB  protocol. 

•  We  expect  to  recruit  5  patients  (~2/year)  for  participation  in  the  study. 

0%  complete.  We  cannot  begin  patient  recruitment  until  protocol  is  approved  by 
Mayo  Clinic  IRB  and  HRPO. 

Subtask  2.2:  Micro-electrode  array  implantation  (Months  4  -  30) 

•  Implantation  of  one  micro-electrode  array  in  either  the  median,  radial,  or  ulnar 
nerve  of  each  patient  will  be  performed. 

Milestone  #1:  Implantation  of  first  patient.  (Months  6) 

Subtask  2.3:  Micro-electrode  array  explantation  (Months  9  -  36) 

•  Explantation  of  the  micro-electrode  array  will  be  performed  at  the  completion  of 
the  study  (30  -  90  days  post-implantation). 

0%  complete.  We  cannot  implant  or  explant  first  patient  until  protocol  is  approved 
by  Mayo  Clinic  IRB  and  HRPO. 


Major  Task  3:  Recording  of  isolated  action  potentials  or  multi-action  potential  activity 
from  residual  peripheral  nerve  while  patient  intends  movements  of  amputated  hand/arm 

Subtask  3.1:  Mapping  of  neural  activity  (Months  4  -  36) 


Patients  will  be  asked  to  intend  a  number  of  individual  finger  and  multiple  finger 
flexion,  extension,  adduction,  and  abduction  movements  of  their  amputated  hand 
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by  mimicking  computer  controlled  movements  of  the  virtual  prosthetic  hand. 
Similarly,  they  will  be  asked  to  make  pronation-supination  forearm  movements;  and 
flexion,  extension,  adduction,  and  abduction  movements  of  their  wrist. 

•  The  spatio-temporal  patterns  of  action  potential  firing  evoked  in  the  efferent  fibers 
of  the  nerve  will  be  recorded  with  the  micro-electrode  array  during  these  intended 
movements.  We  will  map  the  different  intended  movements  onto  the  neural 
activity  recorded  on  the  electrodes  of  the  micro-electrode  array. 

Milestone  #2:  chronic  electrophysiological  recording  from  first  patient  (Month  6) 

25%  complete.  The  infrastructure  for  performing  the  electrophysiological  recordings  and 
nerve  mappings  is  in  place  and  validated. 

Subtask  3.2:  Offline  analysis  and  decoding  of  movements  (Months  6  -  36) 

•  Analyze  the  data  recorded  during  sub-task  3.1  to  determine  if  the  neural  activity  can 
accurately  predict  the  movements  being  intended. 

50%  complete.  The  workstation  computer  and  data  server  have  been  setup  in  the 
Goldwater  Computing  Center,  and  the  offline  analysis  code  have  been  implemented. 

Subtask  3.3:  Online  (real-time)  decoding  of  movements  (Months  6  -  36) 

•  Using  data  collected  during  subtask  3.1  we  will  train  an  online  decode  algorithm  and 
then  provide  the  patient  real-time  control  over  the  virtual  prosthetic  hand. 

Milestone  #3:  real-time  control  of  multiple  degree  of  freedom  virtual  prosthetic  had  in 
first  patient  (Month  9) 

50%  complete.  Computers  for  real-time  computer  analysis  and  control  of  the  virtural 
prosthetic  hand  have  been  implemented  in  the  patient  cart.  Real-time  data  acguisition, 
analysis,  and  control  of  the  virtual  prosthetic  hand  have  be  validated. 


Major  Task  4:  Evaluation  of  somatosensory  perceptions  evoked  by  electrical  micro¬ 
stimulation  of  the  implanted  nerve  via  the  micro-electrode  array 

Subtask  4.1:  Topographical  mapping  and  subjective  description  of  evoked  perceptions 
(Months  6  -  36) 

•  We  will  perform  electrical  micro-stimulation  at  various  currents  on  each  electrode  in 
the  micro-electrode  array  in  order  to  determine  the  minimum  current  needed  to 
consistently  evoke  a  sensory  perception. 

Subtask  4.2:  Spatial  two-point  discrimination  (Months  6  -  36) 


Using  super-threshold  micro-stimulation  levels  obtained  in  subtask  4.1,  we  will 
determine  if  micro-stimulation  on  pairs  of  electrodes  with  differing  inter-electrode 
spacing  evoke  a  single  perception  or  two  spatially  distinct  perceptions. 
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Subtask  4.3:  Modulation  of  evoked  perceptions  (Months  6  -  36) 

•  We  will  determine  if  changes  in  micro-stimulation  parameters  result  in  modulation 
of  the  evoked  sensory  perceptions.  We  will  provide  modulation  of  micro-stimulation 
frequency  using  input  from  tactile  sensors  such  as  would  be  used  in  a  prosthetic 
hand,  i.e.  micro-stimulation  frequency  would  be  modulated  by  the  amount  of 
pressure  on  a  fingertip  tactile  sensor. 

Milestone  #4:  Completion  of  studies  and  longitudinal  analysis  in  all  6  patients. 

Preparation  of  manuscript  for  publication  in  scientific  literature  (Month  36) 

25%  complete.  The  infrastructure  for  performing  micro-stimulation  to  perform 

electrophysiological  recordings  and  nerve  mappings  is  in  place  and  validated. 

What  was  accomplished  under  these  goals? 

We  have  been  making 
progress  on  activities  and 
objectives  related  to  1) 
protocol  preparation,  2) 
improving  the  micro-electrode 
neural  interface,  3)  installing 
and  validating  the  human 
patient  care  for  data 
acquisition  and  micro¬ 
stimulation,  and  4)  improving 
the  virtual  reality  prosthetic 
hand  and  patient  environment 
and  decoding  algorithms. 

Data  from  an  earlier  human 
study  reveal  challenges  with 
using  rigid  silicon  arrays  in  an 
active  human  patient 
population.  The  rigid  silicon 
arrays  were  capable  of  making 
good  electrophysiological 
recordings  and  performing  micro-stimulation,  however,  they  are  susceptible  to  crush 
damage.  We  have  designed  a  compliant  poly-LIFE  micro-electrode  array  using  Kevlar-fiber 
and  cracked-gold  materials  that  are  similar  to  electrode  arrays  used  successfully  in  previous 
human  and  animal  studies  that  is  resistant  to  crush  damage  (Figure  1).  In  contrast  to  the 
rigid  silicon  based  electrode  arrays,  the  materials  used  in  the  poly-LIFE  electrode  arrays 
allow  them  to  compliantly  move  with  the  peripheral  nerves  and  resist  mechanical  damage. 


Figure  1  -  CAD  drawings  of  the  poly-LIFE.  Top  panel: 

A  schematic  representation  of  a  poly-LIFE,  consisting 
of  a  Kevlar  filament  core,  clad  with  metal  layers 
(Ti/Au)  and  insulated  with  silicone.  Bottom  panel: 
finished  single  electrode  with  exposed  Au  electrode 
site  (outer  diameter  100  -  200  pM).  Multiple  poly- 
LIFE  fibers  can  be  formed  into  an  array  with  individual 
electrodes  implanted  into  multiple  individual  nerve 
fascicles. 
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This  information  on  the  electrode  array  device 
design  impacted  the  Mayo  Clinic  IRB  and 
HRPO  applications.  Both  the  original  rigid 
silicon  based  electrodes  arrays  and  the  newer 
compliant  Kevlar  and  cracked  gold  based 
electrode  array  designs  needed  to  be  finalized 
prior  to  inclusion  in  the  protocol.  With  the 
micro-electrodes  array  designs  finalized  and 
their  surgical  implantation  techniques 
addressed,  I  can  complete  and  submit  the 
protocol  application  to  the  Mayo  Clinic  IRB 
and  HRPO.  In  parallel  we  will  make  a  formal 
request  to  CDMRP  to  incorporate  the  poly- 
LIFE  arrays  into  the  study. 

We  have  implemented  the  human  patient 
cart  for  performing  electrophysiological 
recordings  and  micro-stimulation  stimulation. 

The  patient  cart  has  been  assembled  and  we 
have  performed  the  systems  integration  on 
the  computers  and  equipment  needed  for 
performing  the  recording  and  stimulation 
experiments  (Figure  2).  The  patient  cart 
contains  all  the  hardware  needed  to  perform 
the  proposed  real-time  virtual  reality  prosthetic  hand  control  and  sensory  feedback 
experiments.  The  cart  has  been  inspected  and  approved  by  the  Biomedical  Engineering 
department  at  the  Mayo  Clinic  for  recordings. 

We  have  implemented  and  improved  the  virtual  reality  environment  for  patient  training  and 
real-time  control  of  the  virtual  prosthetic  hand.  Patients  in  an  earlier  study  reported  a 
mismatch  between  the  perceived  position  of  their  phantom  hand  and  the  visual  position  of 
the  virtual  reality  prosthetic  hand.  This  interfered  with  their  ability  to  control  the  virtual 
reality  hand  and  their  embodiment  of  the  prosthetic  hand.  In  order  to  improve  patient 
control  during  real-time  decoding  of  neural  signals  and  patient  embodiment  of  the 
prosthetic  hand  we  have  improved  the  virtual  reality  interface.  We  have  made  the  virtual 
reality  prosthetic  hand  environment  fully  immersive  using  Occulus  Rift  goggles.  This 
immersive  environment  allows  the  patients  to  control  a  virtual  reality  hand  mapped  onto 
their  residual  arm.  During  training  the  control  of  the  virtual  prosthetic  hand  is  provided  by 
tracking  movement  of  the  patient's  intact  hand.  The  movements  of  the  intact  hand  can  be 


Figure  2  -  patient  cart.  Recordings  and 
stimulation  systems,  and  computers 
for  real-time  experimental  control  and 
decoding  of  finger  position. 
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monitored  by  an  infra-red  tracker  that  is 
used  in  a  portable  system  that  can  be 
taken  home  by  the  patients  from  practice 
prior  to  array  implantation.  By  using 
tracking  of  the  patients'  intact  hand  prior 
to  implantation  of  the  electrode  array  we 
will  be  able  to  establish  an  accurate 
mapping  of  the  visual  input  of  the  position 
of  the  prosthetic  hand  onto  the  felt 
position  of  the  patient's  phantom  hand. 
Additionally,  practice  with  the  virtual 
prosthetic  hand  immersive  virtual  reality 
environmental  will  enable  patients  to  "re¬ 
activate"  their  phantom  limb  prior  to 
array  implantation  and  therefor  increase 
the  likelihood  or  recording  neural  activity 
correlated  with  hand  control.  After  the 
micro-electrode  arrays  have  been 
implanted  and  during  real-time  decoding 
experiments,  hand  position  will  be 
monitored  using  a  high-speed  and  high- 
resolution  motion  capture  system,  which 
will  allow  precise  mapping  of  the  neural 
signals  onto  finger  and  hand  position 
(Figure  3). 

We  have  improved  on  past  decoding 
efforts  using  machine  learning  algorithms 
based  on  multivariate  time  and  frequency 
domain  features.  Patients  and 
rehabilitation  specialists  have  informed  us 
that  very  high  performing  decoding 
algorithms  are  critical  as  even  a  few 
percent  error  translates  into  an 
unacceptable  failure  rate  in  object 
manipulation,  e.g.  dropping  a  utensil  5% 
of  the  time.  The  machine  learning 
algorithms  are  being  implemented  on  the 
patient  cart  for  use  in  the  real-time 
control  of  the  virtual  prosthetic  hand 
experiments. 


Figure  3  -  Top  panel)  A  student  is  shown 
wearing  the  Occulus  Rift  goggles  and 
infrared  hand  tracking  system.  The  screen 
shows  the  position  of  the  students  hand's 
captured  by  tracking  systems  and  displayed 
to  her  by  the  googles.  Patients  will  be  able 
to  take  this  system  home  in  order  to 
practice  the  tasks  and  improve  the  "re¬ 
activation"  of  their  phantom  limb.  Middle 
panel)  Reflective  markers  are  placed  on  the 
hand  to  allow  precis  tracking  of  hand 
movements.  Bottom  pannel)  The  position 
of  the  wrist  and  finger  joints  as  captured  by 
the  high-reolution  motion  capture  system, 
system. 
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The  Goals  that  have  been  delayed  are  predicated  on  electrode  array  design  and  fabrication. 
Due  to  the  device  failures  with  the  rigid  silicon  based  electrodes  used  in  previous  studies  we 
have  been  very  carefully  considering  the  path  forward  in  consultation  with  device 
fabrication  and  our  clinical  partners.  Although  the  rigid  silicon  arrays  appear  to  have  a  very 
challenging  design  issued  due  to  their  rigidity  and  brittleness,  they  may  be  appropriate  for 
use  in  acute  studies  in  a  few  patients  in  order  to  establish  important  prosthetic  control  and 
sensory  feedback  design  parameters.  The  use  of  compliant  micro-electrode  arrays  will  likely 
address  this  issue  and  provide  path  forward  to  long-term  patient  peripheral  nerve 
interfaces. 

What  opportunities  for  training  and  professional  development  has  the  project  provided? 

Nothing  to  Report 

How  were  the  results  disseminated  to  communities  of  interest? 

The  development  of  the  improved  virtual  reality  environment  for  patient  training  and 
improved  decoding  algorithms  have  been  disseminated  through  presentations  at  scientific 
meetings,  invited  seminars,  and  television  news  programs. 

What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 

Early  in  the  next  reporting  period  we  will  obtain  Mayo  Clinic  IRB  and  HRPO  approvals  and 
implant  the  first  patient.  If  the  poly-LIFE  arrays  cannot  be  fabricated  by  the  time  the 
protocol  is  approved  the  first  patient(s)  will  be  implanted  with  the  rigid  silicon  based 
electrode  arrays.  As  the  compliant  ploy-LIFE  arrays  are  available  we  will  switch  implanting 
them.  All  of  the  infrastructure  for  performing  the  proposed  work  is  in  place,  so  once 
protocol  approval  is  obtained  all  of  the  goals  and  milestones  should  occurred  at  the 
projected  intervals. 

4.  IMPACT: 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

The  development  of  the  immersive  virtual  reality  environment  will  enable  new  techniques 
and  measurements  of  patient  rehabilitation.  We  will  be  able  to  have  patient  practice  with 
their  virtual  reality  prosthetic  hand  prior  electrode  implantation.  This  will  likely  increase  and 
improve  the  quality  of  the  neural  signals  present  in  the  residual  nerve  and  thus  improve  the 
control  over  the  prosthetic  hand  once  the  electrode  array  is  implanted. 

By  using  machine  learning  algorithms  that  utilize  both  time  and  frequency  domain  features 
we  have  improved  the  performance  prosthetic  control.  To  date  control  of  advanced 
prosthetic  hands  have  utilized  linear  decoding  algorithms  that  do  not  incorporate  the 
knowledge  the  most  neural  signals  encode  information  in  a  nonlinear  manner.  These 
improved  algorithms  will  likely  increase  patient  acceptance  of  advance  prosthetic  hands,  as 
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even  a  few  percent  improvement  in  control  translates  into  fewer  unwanted  movements, 
e.g.  dropping  a  help  object. 

What  was  the  impact  on  other  disciplines? 

The  development  and  use  of  compliant  poly-LIFE  arrays  may  enable  multiple 
applications  that  necessitate  long-term  peripheral  nerve  interfaces:  bowel/bladder 
control  in  paralyzed  patients,  regulated  hypertension,  and  treatment  of  epilepsy.  The 
machine  learning  algorithms  we  have  developed  for  decoding  neural  signal  in  residual 
peripheral  nerves  can  also  be  applied  to  neural  signals  recorded  from  the  central 
nervous  system  for  application  to  provide  control  for  severely  paralyzed  patients. 

What  was  the  impact  on  technology  transfer? 

An  invention  disclosure  on  the  use  and  surgical  placement  of  the  compliant  electrode 
arrays  was  filed  with  Arizona  Technology  Enterprises  (AzTE). 

What  was  the  impact  on  society  beyond  science  and  technology? 

Nothing  to  Report 
5.  CHANGES/PROBLEMS: 

Changes  in  approach  and  reasons  for  change 

Since  the  inception  of  the  current  project  I  have  learned  of  failure  modes  of  the  rigid 
silicon  electrode  arrays  originally  planned  on  using  for  the  project  and  encountered 
difficulties  with  my  surgical  collaborator.  Most  of  these  issues  only  came  to  light  as  the 
CDMRP  funded  current  project  was  starting  up. 

Surgical  and  electrode  array  Issues:  Dr.  Hutchinson  the  surgeon  I  had  originally  planned 
on  collaborating  with  for  this  project,  was  also  the  surgeon  on  a  DARPA  funded  project 
of  which  I  was  involved.  As  part  of  the  DARPA  funded  work  he  implanted  an  array  in 
NHP.  This  surgery  resulted  in  the  nerve  in  NHP  being  destroyed  and  the  animal  having  to 
be  sacrificed.  During  the  DARPA  funded  project  he  also  harvested  tissue  from  a  patient 
without  first  obtaining  consent.  Given  the  surgical  failures  and  what  I  perceive  as 
questionable  ethical  behavior,  I  cannot  in  good  conscience  collaborate  further  with  Dr. 
Hutchinson  and  will  be  collaborating  only  at  the  Mayo  Clinic  in  Scottsdale.  We  will  be 
formally  requesting  a  cancelation  of  the  University  of  Utah  subcontract.  In  a  previous 
DARPA  funded  project  four  patients  were  implanted  at  the  Univ.  of  Utah  with  the 
electrode  arrays  I  was  originally  planning  on  using  for  the  current  project.  I  would 
categorize  two  of  these  patient  implantations  as  failures.  One  electrode  array  failed 
after  a  week  of  implantation  due  to  the  array/wire  being  crushed/broken  by  the  patient 
likely  during  performance  of  their  job.  The  second  array  failure  occurred  a  few  day  after 
implantation  likely  due  to  reference  and/or  ground  wire  breakage  and/or  improper 
surgical  placement  of  reference/ground  leads.  Given  the  high  failure  rate  observed,  we 
are  planning  on  switching  to  a  more  compliant  electrode  array  technology  that  has  a 
better  chance  of  serving  as  platform  for  the  long-term  control  of  a  prosthetic  hand.  We 
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will  be  making  a  formal  request  to  include  the  compliant  electrode  array  in  the 
proposed  work. 

Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

Actual  delays: 

Electrode  array  failures. 

Resolution  -  Transition  from  rigid  silicon  arrays  to  compliant  Kevlar-fiber  and 
cracked-gold  poly-LIFE  arrays. 

Surgical  issues  and  challenges. 

Resolution:  Perform  all  patient  work  at  the  Mayo  Clinic. 

Failure  in  communicating  well  with  CDMRP: 

Resolution:  This  has  primarily  be  due  to  a  significant  health  issue  with  my  wife  (I 
can  provide  details  and  documentation  if  required).  She  has  been  undergoing 
treatment  for  the  past  few  months  and  is  improving  significantly.  I  am  confident 
that,  barring  and  major  setbacks,  I'll  be  able  to  be  responsive  in  a  timely  manner 
in  the  future. 

Anticipated  delays:  None 

Changes  that  had  a  significant  impact  on  expenditures 

No  to  report. 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards,  and/or 
select  agents 

None  to  report. 

Significant  changes  in  use  or  care  of  human  subjects 

None  to  report. 

Significant  changes  in  use  or  care  of  vertebrate  animals. 

Not  applicable. 

Significant  changes  in  use  of  biohazards  and/or  select  agents 

Not  applicable. 

6.  PRODUCTS:  List  any  products  resulting  from  the  project  during  the  reporting  period.  If  there  is 
nothing  to  report  under  a  particular  item,  state  "Nothing  to  Report. " 

Publications,  conference  papers,  and  presentations 

Greger  B,  Decoding  finger  movements  from  peripheral  nerve  signals.  2015  4th  ASU  Rehabilitation 
Robotics  Workshop 
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Greger  B,  Evoking  sensory  percepts  using  electrical  micro-stimulation  of  the  nervous  system. 
2015  invited  seminar  SUNY  Upstate  Medical  University 

Greger  B,  Evoking  sensory  percepts  using  electrical  micro-stimulation  of  the  nervous  system. 
2015  invited  seminar  Rice  University 

Barton  C,  Padmanaban  S,  Kellis  S,  House  P,  GREGER  B,  Extracting  high-frequency  features  of 
neural  activity  from  pECoG  surface  electrodes.  2015  Society  for  Neuroscience  Abstract  522.17 

Oswalt  D,  Kellis  S,  House  P,  Greger  B,  Multiclass  support  vector  machine  decoding  of  spoken 
words  from  micro-electrocorticography  recordings  over  Wernicke's  area  and  face  motor  cortex. 
2015  Society  for  Neuroscience  Abstract  522.18 

Inventions,  patent  applications,  and/or  licenses 

Invention  disclosure:  TUFT  peripheral  nerve  interface:  a  compliant  micro-electrode  array  for 
recording  electrophysiological  signals  and  micro-stimulating  neural  tissue.  5/30/2014 

Other  Products 

None  to  report. 

7.  PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 
What  individuals  have  worked  on  the  project? 
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Bradley  Greger 
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Funding  Support: 
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6 

Contribution  to  Project: 
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Funding  Support: 
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2 
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Funding  Support: 

CDMRP 
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1 

Contribution  to  Project: 

Cody  Barton  has  been  investigating  the  use  of  high-frequency  power 
as  an  adjunct  to  action  potential  recordings  for  controlling  prosthetic 
hands 

Funding  Support: 
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Nearest  person  month  worked: 
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vector  machines  for  decoding  neural  signals 
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Funding  Support: 
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and  constrained  coding. 

Funding  Support: 
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Davis  T,  Wark  HAC,  Hutchinson  DT,  Warren  DJ,  O'Neill  III  K,  Scheinblum  T,  Clark  GA,  Normann 
RA,  Greger  B,  Restoring  motor  control  and  sensory  feedback  in  people  with  upper  extremity 
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amputations  using  arrays  of  96  microelectrodes  implanted  in  the  median  and  ulnar  nerves. 
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Paper  provides  information  on  earlier  study  and  support  for  proposed  changes  to  current  work. 
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Restoring  motor  control  and  sensory  feedback  in  people  with 
upper  extremity  amputations  using  arrays  of  96  microelectrodes 
implanted  in  the  median  and  ulnar  nerves 

T  S  Davis12*,  H  A  C  Wark1*,  D  T  Hutchinson3,  D  J  Warren1,  K  O’Neill  III1,  T 
Scheinblum1,  G  A  Clark1,  R  A  Normann1,  and  B  Greger1’4 
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'Department  of  Bioengineering,  University  of  Utah,  Salt  Lake  City,  UT,  84112  USA 
department  of  Neurosurgery,  University  of  Utah,  Salt  Lake  City,  UT,  84132  USA 
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E-mail:  bradley.greger@asu.edu 

Abstract 

Objective:  An  important  goal  of  neuroprosthetic  research  is  to  establish  bidirectional  communication 
between  the  user  and  new  prosthetic  limbs  that  are  capable  of  controlling  >  20  different  movements.  One 
strategy  for  achieving  this  goal  is  to  interface  the  prosthetic  limb  directly  with  efferent  and  afferent  fibres 
in  the  peripheral  nervous  system  using  an  array  of  intrafascicular  microelectrodes.  This  approach  would 
provide  access  to  a  large  number  of  independent  neural  pathways  for  controlling  high  degree-of- freedom 
prosthetic  limbs,  as  well  as  evoking  multiple-complex  sensory  percepts. 

Approach:  Utah  Slanted  Electrode  Arrays  (USEAs,  96  recording/stimulating  electrodes)  were  implanted 
for  30  days  into  the  median  (subject  1-M,  31  years  post-amputation)  or  ulnar  (subject  2-U,  1.5  years 
post-amputation)  nerves  of  two  amputees.  Neural  activity  was  recorded  during  intended  movements  of 
the  subject’s  phantom  fingers  and  a  linear  Kalman  filter  was  used  to  decode  the  neural  data. 
Microelectrode  stimulation  of  varying  amplitudes  and  frequencies  was  delivered  via  single  or  multiple 
electrodes  to  investigate  the  number,  size  and  quality  of  sensory  percepts  that  could  be  evoked.  Device 
performance  over  time  was  assessed  by  measuring:  electrode  impedances,  signal-to-noise  ratios, 
stimulation  thresholds,  number  and  stability  of  evoked  percepts. 

Mam  results:  The  subjects  were  able  to  proportionally,  control  individual  fingers  of  a  virtual  robotic 
hand,  with  13  different  movements  decoded  offline  (r=0.48)  and  two  movements  decoded  online. 
Electrical  stimulation  across  one  USEA  evoked  >80  sensory  percepts.  Varying  the  stimulation 
parameters  modulated  percept  quality.  Devices  remained  intrafascicularly  implanted  for  the  duration  of 
the  study  with  no  significant  changes  in  the  signal-to-noise  ratios  or  percept  thresholds. 

Significance:  This  study  demonstrated  that  an  array  of  96  microelectrodes  can  be  implanted  into  the 
human  peripheral  nervous  system  for  up  to  1  month  durations.  Such  an  array  could  provide  intuitive 
control  of  a  virtual  prosthetic  hand  with  broad  sensory  feedback. 

1.  Introduction 

The  volitional  control  of  movement  involves  a  complex  and  integrated  hierarchy  of  sensory-motor  neural 
systems.  Arrays  of  microelectrodes  implanted  at  various  levels  in  this  hierarchy  have  been  used  to  record 
spatiotemporal  patterns  of  neural  activity  and  for  correlation  of  these  patterns  with  sensory  stimulation  and 
motor  behaviours.  Early  efforts  to  obtain  volitional  control  signals  from  the  nervous  system  were  conducted 
in  non-human  primates  by  decoding  neural  activity  patterns  recorded  with  microelectrodes  implanted  in  the 
motor  cortex  [1,  2,  3,  4,  5,  6].  Researchers  have  begun  translating  these  efforts  to  human  subjects,  and 
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electrode  arrays  implanted  into  the  motor  cortex  of  people  with  paralysis  have  provided  the  unidirectional 
communication  needed  for  the  volitional  control  of  external  devices  such  as  the  two-dimensional 
movements  of  computer  cursors  [7,  8,  9]  and  the  robotic  manipulation  of  objects  in  three-dimensional  space 
[10].  Although  the  central  nervous  system  is  an  effective  implantation  site  for  people  with  paralysis,  less 
invasive  approaches  are  warranted  for  people  with  amputations,  such  as  peripheral  nerve  targeted 
reinnervation  [11,  12]  or  peripheral  nerve  electrode  implantation  [13,  14,  15,  16,  17,  18,  19,  20,  21,  22], 

The  current  state-of-the-art  for  people  with  upper-limb  amputations  are  mechanically  or  myoelectrically 
controlled  prosthetic  arms  with  few  degrees-of-freedom  (DOF)  that  do  not  provide  sensory  feedback  [23]. 
Next  generation  prosthetic  arms  are  being  developed  with  upwards  of  26  DOF  [24,  25]  and  with  embedded 
sensors  intended  to  be  used  to  provide  sensory  feedback  to  the  user  [  1 4, 2 1 , 26] .  Several  strategies  are  being 
investigated  for  providing  sensory  feedback  from  these  new  prosthetic  limbs  including  surface  vibrators 
[26],  and  extraneural  [18,  19,  22,  27]  or  intraneural  [21,  28,  29,  30,  31]  electrodes.  The  number  of 
extra/intraneural  electrodes  (or  ‘contacts’)  implanted  into  each  human  peripheral  nerve  has  been  increasing, 
with  previous  studies  having  used:  one  [14,  17],  six  [32],  eight  [22,  33],  sixteen  [21],  or  twenty  [30] 
electrodes  to  established  bidirectional  communication  with  a  subject’s  peripheral  nervous  system. 

Modem,  high-DOF  bidirectional  prosthetic  arms  and  hands  may  require  numerous  microelectrodes  to 
obtain  sufficiently  independent  control  and  feedback  signals.  Motor  intent  has  been  decoded  online  and 
offline  [14,  20,  21,  32]  with  control  of  up  to  three  different  movements  from  offline  decodes  [14,  20,  21], 
and  sensory  feedback  has  been  provided  via  electrical  stimulation  [14,  15,  17,  21,  22,  26,  33,  34]  with  the 
evocation  of  a  maximum  of  nine  different  percepts  for  a  single  neural  implant  [21]. 

We  report  herein  an  approach  for  the  control  of  prosthetic  limbs  using  an  array  of  96  microelectrodes,  the 
Utah  Slanted  Electrode  Array  (USE A)  [29],  which  increases  the  number  of  electrodes  that  have  been  used 
to  interface  with  a  peripheral  nerve  by  80  more  electrodes  per  nerve  over  previously  investigated  neural 
interfaces  [21,  22],  This  work  was  conducted  in  two  subjects  who  had  previously  undergone  transradial 
amputations.  Each  subject  had  a  microelectrode  array  implanted  in  one  of  their  transected  peripheral  nerves 
in  order  to  selectively  access  individual  or  small  groups  of  motor  and/or  sensory  fibres.  Selective 
spatiotemporal  neural  activity  patterns  were  recorded  when  subjects  were  asked  to  make  volitional 
movements  of  their  phantom  fingers.  Closed-loop  neural  control  of  multiple  cursors  or  virtual  robotic 
fingers  was  achieved  in  each  subject  with  two  DOF  control  achieved  online  for  both  subjects. 

To  our  knowledge,  this  is  the  first  peer-reviewed  publication  that  investigates  both  stimulation  and 
recording  of  human  peripheral  nerves  with  high-count  (96  electrodes)  intraneural  electrodes.  The  work 
presented  in  this  manuscript  increases  the  number  of  movements  that  were  decoded  from  human  peripheral 
nerve  from  3  different  grasp  movements,  equivalent  of  1  DOF  [20,  21]  to  13  different  movements, 
equivalent  to  8  DOF.  The  maximum  number  of  sensory  percepts  evoked  by  a  single  peripheral  neural 
interface  was  previously  eight  [22]  and  in  this  study  we  increase  that  number  to  a  maximum  of  86  percepts 
(number  of  percepts  in  a  single  stimulation  session)  with  an  average  of  8 1  percepts  over  the  study  duration. 
This  work  demonstrates  that  an  array  of  microelectrodes  implanted  into  residual  nerves  of  patients  with  a 
forearm  amputation  allows  for  selective  access  of  both  sensory  and  motor  nerve  fibres.  These  results  further 
suggest  that  intuitive  and  dexterous  control  of  prosthetic  fingers  with  sensory  feedback  can  be  provided  for 
future  bidirectional  prosthetic  limbs  using  an  array  of  intrafascicularly  implanted  microelectrodes. 
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2.  Methods 

This  study  was  approved  by  the  University  of  Utah  Institutional  Review  Board,  the  Salt  Lake  City  Veterans 
Affairs  Hospital  Research  and  Development  Service  Center  and  the  U.S.  Federal  Defense  Advanced 
Research  Projects  Agency. 

2.1.  Pre-study  enrolment  period 

Two  volunteers  with  previous  transradial  amputations  participated  in  this  study  and  underwent  implantation 
for  a  one  month  period  with  a  USEA  into  their  median  nerve  (Subject  1-M,  Median  nerve  implant,  3 1  years 
post-amputation)  or  ulnar  nerve  (Subject  2-U,  Ulnar  nerve  implant,  1.5  years  post-amputation).  Potential 
volunteers  were  evaluated  for  the  extent  by  which  they  perceived  that  they  were  able  to  make  specific 
movements  with  their  phantom  fingers.  The  duration,  frequency,  and  intensity  of  phantom  limb  sensations, 
including  pain,  were  documented  in  the  patient’s  journal  throughout  the  study.  Multiple  phantom 
movements  mediated  by  median  and  ulnar  nerve  activity  were  evaluated  in  each  volunteer,  and  the  extent 
of  their  ability  to  move  their  fingers  was  noted  from  their  descriptions  (e.g.  Subject  1-M  had  ‘a  very  fluid 
hand’  but  Subject  2-U’s  phantom  fingers  were  often  ‘clenched  tight’).  After  selection  for  inclusion  in  the 
study,  volunteers  were  then  given  a  mirror  box  and  specific  exercises  to  perform  in  order  to  strengthen  their 
perceived  ability  to  move  the  digits  on  their  phantom  hands.  Volunteers  were  asked  to  document  any 
changes  in  the  perceived  control  of  their  phantom  hand,  and  if  any  decreased  control  or  unpleasant  sensation 
was  experienced,  the  volunteers  would  have  been  asked  to  discontinue  use  of  the  mirror  box  (none  of  the 
volunteers  experienced  such  changes).  Prior  to  enrolment,  volunteers  underwent  a  psychosocial  evaluation 
in  order  to  determine  if  any  underlying  psychological  conditions  were  present  which  would  have  excluded 
them  from  the  study. 

2.2.  Nerve  electrode  arrays 

The  USEA  (Blackrock  Microsystems,  Salt  Lake  City,  UT,  USA)  has  been  described  elsewhere  [29]  and 
will  only  be  briefly  described  herein.  The  array  consists  of  100  electrodes  (96  recording/stimulating 
electrodes,  4  electrodes  wired  as  unused  backup  reference  electrode)  with  lengths  ranging  from  0.5  to  1.5 
mm  (10  by  10  grid  with  400  pm  spacing)  that  project  out  from  a  4  x  4  x  0.3  mm3  substrate.  Lead  wire 
lengths  from  the  connector  (custom-built  item  compatible  with  the  ZIF  Clip  96,  Tucker  Davis  Technologies, 
Inc.,  Alachua,  FL,  USA)  to  the  array  were  configured  for  implantation  into  the  upper  limb  nerves  at  a 
transradial  location.  Lead  wires  included:  1)100  lead  wires  (~9  cm  long)  with  96  of  the  wires  bonded  to 
electrodes  used  for  recording  and  4  bonded  to  backup  reference  electrodes,  2)  two  low-impedance  platinum 
reference  wires  (~8.5  cm  long)  connected  to  the  reference  channel,  and  3)  two  low-impedance  platinum 
ground  wires  (~7.5  cm  long)  connected  to  the  recording  amplifier  ground  and  stimulation  return. 
Approximately  1 0  mm  of  the  distal  ends  of  the  reference  and  ground  wires  were  twisted  and  looped  back, 
with  distal  ends  secured  onto  the  proximal  reference  or  ground  wires  by  coating  with  silicone  (MED-421 1, 
NuSil  Technology  LLC,  Carpinteria,  CA,  USA).  Thus,  each  loop  consisted  of  two  wires  (two  reference  or 
two  ground  wires),  and  the  distal  ends  of  these  loops  were  de-insulated  to  reduce  their  impedance.  The 
lengths  of  the  reference  and  ground  wires  were  measured  from  the  base  of  the  connector  to  the  end  of  the 
distal  loop.  The  reference  and  ground  wires  were  placed  inside  the  nerve  wrap  (see  section  2.3  surgical 
procedures)  near  the  electrode  array. 

A  Neuroport  data  acquisition  system  (Blackrock  Microsystems,  Salt  Lake  City,  UT,  USA)  was  used  to 
measure  the  electrode  impedances.  A  sinusoidal  current  at  1  kHz  was  passed  through  a  reference  electrode, 
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and  impedance  was  simultaneously  computed  on  all  electrodes.  Before  implantation,  the  electrodes  on  each 
device  had  an  average  (mean±  std)  impedance  of  75  ±  57  k£2  (Subject  1-M)  and  90  ±  28  kD  (Subject  2-U). 
Working  electrodes  were  defined  as  electrodes  with  an  impedance  <  500  kQ. 

2. 3.  Surgical  procedures 

The  distal  nerve  end  was  exposed  and  the  implant  site  was  selected  such  that  it  was  distal  to  any  branching 
points  and  proximal  to  the  transitional  zone  adjacent  to  the  neuroma  (approximately  5-10  mm  from  the 
neuroma).  The  USEA  was  then  passed  transcutaneously  via  a  trocar  (7-8  mm  diameter)  down  to  the  exposed 
implantation  site.  A  metal  platform  was  placed  underneath  the  nerve,  along  with  a  high  visibility 
background  and  a  reconstituted  organic  nerve  wrap  (AxoGuard  Nerve  Wrap,  AxoGen  Inc.,  Alachua,  FL, 
USA).  The  lead  wires  were  then  sutured  (8-0  nylon)  to  the  epineurium  (~5  mm  from  the  base  of  the 
electrode  array).  The  USEAs  were  inserted  with  a  pneumatic  insertion  device  [31]  that  was  hand-held  by 
the  surgeon.  The  implanted  USEA,  reference  wires,  ground  wires,  and  nerve  were  contained  within  the 
organic  nerve  wrap,  which  was  closed  snuggly  around  the  implant  site  with  titanium  vascular  clips 
(Supplementary  Figure  1 ).  The  organic  nerve  wrap  was  sutured  (8-0  nylon)  to  the  epineurium  proximal  and 
distal  to  the  array  site  to  prevent  movement  of  the  wrap  along  the  nerve. 

The  percutaneous  site  was  dressed  with  a  1”  diameter,  clorohexadine  antibacterial  patch  (Biopatch,  Ethicon 
Inc.,  Johnson  &  Johnson,  New  Brunswick,  NJ,  USA).  In  Subject  2-U,  the  connector  was  sutured  down  to 
the  skin  to  prevent  stress  on  the  lead  wires  due  to  movement  of  the  connector.  The  entire  percutaneous  site 
was  layered  with  gauze  and  covered  with  a  breathable  and  waterproof  transparent  dressing  (Tegaderm,  3M 
Healthcare,  St.  Paul  MN,  USA).  The  gauze  and  film  dressing  was  changed  during  each  experimental  session 
and  antibacterial  patches  were  replaced  every  7-10  days.  To  decrease  the  inflammatory  process  and 
potentially  assist  in  enhanced  signal  quality  over  time,  subjects  were  given  dexamethasone  (0.1  mg/kg  IV, 
Mylan  Institutional  LLC,  Rockford,  IL)  intraoperatively  after  removal  of  the  tourniquet  and  minocycline 
(100  mg  BID,  Watson  Pharmaceuticals,  Parsippany,  NJ)  2  days  prior  and  for  5  days  after  surgery  [35,  36]. 

At  the  end  of  the  one  month  experimental  period,  each  subject  underwent  explantation  of  the  USEAs  under 
general  anaesthesia.  The  percutaneous  wires  were  cut  at  the  level  of  the  skin  and  the  entire  limb  was  then 
prepared  for  surgery.  The  implant  side  was  exposed  and  the  lead,  reference,  and  ground  wires  were  cut 
adjacent  to  the  organic  nerve  wrap  containment  system  and  all  wires  were  removed.  The  entire  implant  site 
and  neuroma  was  excised.  The  new  nerve  end  was  then  sutured  deep  in  the  surrounding  musculature 
according  to  standard  surgical  procedures  for  neuroma  excisions. 

2. 4.  Experimental  sessions 

Two  hour  experimental  sessions  were  performed  on  an  average  of  three  times  per  week.  The  time  was 
limited  by  subject  availability  or  their  willingness  to  continue  testing  (Subject  1-M  underwent  12  total 
experimental  sessions:  6  electrophysiological  recording  sessions  and  8  microstimulation  sessions;  Subject 
2-U  underwent  a  total  of  14  experimental  sessions:  13  electrophysiological  recording  sessions  and  8 
microstimulation  sessions).  All  experimental  sessions  were  recorded  with  a  video  camera,  which  was  time- 
stamped  to  the  neural  recording  or  stimulation  data. 

2.5.  Neural  recordings,  decoding  and  instrumentation 

Neural  signals  were  amplified  and  recorded  using  active  head-stage  cables  (ZIF-Clip  96,  Tucker  Davis 
Technologies,  Inc.,  Alachua,  FL,  USA)  that  connected  to  a  custom-built  interconnect  board  used  to  interface 
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the  ZIF-Clip  cable  with  the  Neuroport  data  acquisition  system.  The  continuous  neural  signals  were  band¬ 
pass  filtered  with  cutoff  frequencies  of  0.3  Hz  (lst-order  high-pass  Butterworth  filter)  and  7500  Hz  (3rd- 
order  low-pass  Butterworth  filter)  and  sampled  at  30  kHz.  Online  multi-unit  activity  was  extracted  from 
high-pass  filtered  recording  data  (250  Hz  4th-order  Butterworth  filter)  by  setting  a  threshold  using  the  auto 
threshold  setting  in  the  Neuroport  data  acquisition  software  (multiplier  =  3,  threshold  =  multiplier  x  noise 
estimate  of  the  signal).  For  each  of  the  96  neural  recording  channels,  multi-unit  neural  firing  rates  were 
calculated  using  unsorted  spikes  and  a  moving  box-car  average  of  300  ms  with  an  update  period  of  33  ms. 
Offline,  action  potentials  were  isolated  from  the  high-pass  filtered  data  using  commercially  available 
software  (Offline  Sorter  version  3,  Plexon  Inc.,  Dallas,  Texas,  USA).  Signal-to-noise  ratios  (SNR)  were 
calculated  by  dividing  the  mean  peak-to-peak  action  potential  amplitude  by  two  times  the  standard 
deviation  of  the  recorded  noise  [37]. 

2.5.1.  Decoding  neural  signals  and  control  of  virtual  robotic  fingers  or  computer  indicators.  A  standard 
Kalman  filter  was  implemented  to  perform  the  continuous  neural  decodes  [38].  This  algorithm  assumes  a 
linear  relationship  between  the  kinematics  (finger  position)  and  the  neural  data.  For  this  study,  it  was  used 
to  provide  continuous  estimates  of  finger  position  based  on  the  firing  rates  of  multiple  neurons.  Sessions 
began  by  cueing  the  subjects  to  perform  multiple  flexions,  extensions  or  abductions  of  their  individual 
phantom  fingers.  A  total  of  4  movements  were  performed  for  Subject  1-M  (Thumb-Flex,  Index-Flex, 
Middle-Flex,  and  Ring-Flex)  and  13  movements  for  Subject  2-U  (Thumb-Flex,  Thumb-Extend,  Index-Flex, 
Index -Extend,  Index-Abduct,  Middle-Flex,  Middle-Extend,  Ring-Flex,  Ring-Extend,  Ring-Abduct,  Little- 
Flex,  Little-Extend,  Little-Abduct).  These  movements  are  given  acronyms  based  on  the  first  one  or  two 
letters  of  each  word,  which  are  used  in  subsequent  figures  (e.g.,  Thumb-Flex  =  TF).  The  subjects  were 
instructed  to  make  finger  movements  by  either  a  computer  controlled  display  of  indicators  (Subject  1-M) 
or  by  movement  of  virtual  robotic  fingers  [39]  (Subject  2-U)  (Supplementary  Figure  1).  Subject  1-M  held 
a  small  manipulandum  consisting  of  individual,  movable  pads  that  could  be  depressed  by  each  finger  with 
the  subject’s  intact  hand.  The  subject  was  asked  to  mirror  the  movements  made  with  the  subject’s  phantom 
fingers  by  pressing  on  the  manipulandum  pads  with  the  subject’s  intact  fingers.  The  instructed  finger 
position  (instruction  variable)  for  Subject  1-M  was  measured  as  the  continuous  voltage  signal  from  pressure 
sensors  on  the  pads  of  the  manipulandum.  Subject  2-U  was  instructed  to  duplicate  the  movements  of  the 
virtual  prosthetic  hand  using  the  subject’s  phantom  hand.  The  instructed  finger  position  (instruction 
variable)  for  Subject  2-U  was  taken  as  the  software -generated  position  of  the  virtual  prosthetic  finger.  The 
positions  of  the  virtual  prosthetic  fingers  were  generated  using  a  cosine  function,  which  was  normalized 
from  -1  (full  extension/adduction)  to  +1  (full  flexion/abduction).  Multi-unit  firing  rates  from  selected 
electrodes  and  the  movement  instruction  variables  were  then  used  to  train  and  test  the  decode  algorithm  for 
both  online  and  offline  control.  During  online  neural  control  the  subjects  made  finger  movements  to 
computer  generated  targets  while  simultaneously  controlling  multiple  fingers  with  continuous  modulation 
of  finger  position,  i.e.  proportional  control.  A  trial  would  start  when  the  subject  placed  all  prosthetic  fingers 
under  neural  control  at  a  hand-at-rest  position  indicated  by  the  computer  generated  targets.  After  a  few 
hundred  milliseconds,  the  computer  generated  targets  would  shift  positions  and  the  subjects  were  required 
to  match  finger  positions  of  all  prosthetic  fingers  under  neural  control  to  the  positions  of  all  the  computer 
generated  targets  for  at  least  300  ms  and  up  to  3000  ms.  The  targets  would  then  return  to  the  hand-at-rest 
position  and  the  subject  would  have  to  match  the  fingers  to  the  targets  in  order  to  begin  a  new  trial.  For 
example  a  finger  would  be  required  to  move  from  the  hand-at-rest  position  to  a  target  by  flexing  or 
extending  the  finger,  while  simultaneously  the  other  fingers  would  be  required  to  maintain  the  hand-at-rest 
position.  The  finger  required  to  move  the  target  and  the  position  of  the  target  would  be  changed  for  different 
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trials.  This  produced  individuated  proportional  finger  movements  during  simultaneous  neural  control  of 
multiple  fingers  (See  Supplemental  Video  2). 

2.5.2.  Online  neural  decodes.  For  online  decoding,  electrodes  were  selected  based  on  their  ability  to  record 
movement-correlated  action  potentials.  This  selection  process  was  made  using  two  methods:  1)  the 
experimenters  viewed  a  map  of  correlation  coefficients  between  the  instruction  variables  and  the  firing  rates 
on  each  electrode  and  2)  the  experimenters’  subjective  observations  of  the  high-pass  filtered  neural  data  on 
each  electrode  during  the  cued  movements.  Following  electrode  selection,  the  decode  algorithm  was  trained 
on  a  set  of  10  trials  for  each  movement  type.  For  testing,  subjects  were  asked  to  control  a  computer  display 
of  indicators  (Subject  1-M)  or  virtual  prosthetic  fingers  (Subject  2-U)  and  acquire  targets  in  a  trial-based 
format.  Subjects  began  a  trial  by  moving  the  fingers  of  their  phantom  hand  to  a  neutral,  at-rest  position, 
which  moved  the  indicators  or  virtual  prosthetic  fingers  to  a  start  position.  Once  all  fingers  were  at  the  start 
position  for  a  specified  hold  period,  subjects  would  be  shown  the  target(s)  to  which  they  were  to  move  their 
finger(s).  Targets  were  presented  of  varying  diameters  (30%  to  36%  of  full-range  movement)  and  distances 
(40%  to  1 00%  of  full-range  movement)  from  the  starting  point.  In  order  to  correctly  complete  a  trial,  the 
subjects  had  to  acquire  and  hold  the  target(s)  while  maintaining  the  fingers  that  were  not  being  instructed 
to  move  at  their  start  positions.  Start  and  target  hold  periods  varied  from  300  to  5000  ms.  This  task  paradigm 
assessed  the  ability  of  subjects  to  move  the  virtual  fingers/cursors  independently.  Targets  were  placed  at 
various  distances  to  assess  the  ability  of  the  subjects  to  proportionally  control  the  indicators  or  virtual 
fingers. 

2.5.3.  Offline  neural  decodes.  For  offline  decoding,  an  average  of  22  (range  of  13  to  37)  trials  of  each 
movement  type  recorded  during  a  typical  experimental  session  were  analysed.  Electrodes  were  chosen  on 
the  basis  of  the  results  of  a  Wilcoxon  signed-rank  test.  First,  a  ‘baseline-period’  was  defined  as  the  2  s 
period  prior  to  the  onset  of  the  movement  cue,  and  a  ‘movement-period’  was  defined  as  the  2  s  period  after 
the  onset  of  the  cue.  Then,  the  difference  in  median  firing  rates  was  calculated  between  the  ‘movement- 
period’  and  the  ‘baseline-period’  for  all  movement  types  and  trials  recorded  on  each  electrode.  The  null 
hypothesis  was  that  the  data  came  from  a  continuous,  symmetric  distribution  with  a  median  difference  equal 
to  zero  (i.e.,  the  electrode  did  not  record  increased  firing  rates  in  the  movement-period  compared  with  the 
baseline -period).  All  electrodes  for  which  the  null  hypothesis  was  rejected  (p  <  0.001)  with  a  positive 
median  difference  from  baseline  were  kept.  These  electrodes  were  then  sorted  in  order  of  increasing  median 
difference  and  the  top  90%  of  electrodes  were  used  in  the  offline  decode.  After  electrode  selection,  the 
decode  algorithm  was  trained  on  the  first  1 0  trials  for  each  movement  and  tested  on  the  remaining  3  to  27 
trials.  A  Pearson’s  correlation  coefficient  (r)  between  the  instruction  variable  and  the  Kalman  filter  estimate 
of  finger  positions,  calculated  for  the  entire  continuous  sampling  of  data  corresponding  with  the  trained 
movements  for  the  remaining  3-27  testing  trials,  was  used  to  quantify  the  decode  performance. 

2.6.  Neural  stimulation  and  instrumentation 

Current-controlled,  biphasic,  cathodic-first  (without  anodic  bias)  stimulation  [40]  (1Z2,  128-channel 
Tucker-Davis  Technologies  Stimulator,  Inc.,  Alachua,  FL,  USA)  was  delivered  to  individual  or  subsets  of 
electrodes  using  custom  LabVIEW  software  (National  Instruments  Corp.,  Austin,  TX,  USA),  using  the 
platinum  ground  wires  as  a  return.  The  stimulator  had  a  compliance  voltage  of  ±  15V  (LZ48-400,  Tucker- 
Davis  Technologies  Stimulator,  Inc.,  Alachua,  FL,  USA).  Maximum  stimulation  was  limited  by  either:  1) 
the  comfort  level  of  the  subject,  2)  a  perceived  change  in  the  quality  of  the  percept  or  3)  if  the  maximum 
safety  limit  for  delivering  electricity  to  tissue  was  reached  [41].  The  safety  limit  for  injecting  charge  into 
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the  tissue  was  determined  by  measuring  the  maximum  cathodic  voltage  excursion  (with  a  safety  limit  of  - 
0.6  V)  across  the  electrode  and  ground  during  stimulation  [42,  43,  44],  The  voltage  drop  due  to  tissue 
impedance  was  not  subtracted  in  our  calculations,  resulting  in  a  conservative  estimate  of  safe  stimulation 
parameters.  Stimulation  parameters  varied  depending  on  the  objective  of  the  experimental  session.  Pulse 
amplitude,  frequency,  and  train  duration  ranged  from  1-100  pA,  1-320  Hz,  and  0.2-60  s,  respectively.  Pulse 
width  and  inter-phase  interval  were  held  fixed  at  200  ps  and  100  ps. 

2.6.1.  Stimulation  across  different  amplitudes  and  frequencies  (Subject  1-M).  In  Subject  1-M,  stimulation 
experiments  were  focused  on  a  subset  of  electrodes  to  investigate  the  subject’s  ability  to  detect  and 
discriminate  single  and  multiple  sensory  percepts  and  the  effects  of  modulating  stimulation  frequency  on 
sensory  percepts.  A  custom-built  software  interface  (LabVIEW  software,  National  Instruments  Corp., 
Austin,  TX,  USA)  was  used  to  allow  Subject  1-M  to  control  the  amplitude  of  stimulation  (1-100  pA  in  steps 
of  1  pA)  at  a  constant  frequency  (200  Hz)  and  train  duration  (0.2  s)  in  order  to  determine  the  threshold  of 
a  sensory  percept  (sessions  1-8,  post-implantation  days  5-26).  Control  of  frequency  (1-320  Hz  in  1  Hz  steps, 
threshold  amplitudes,  0-60  s  durations)  was  provided  to  the  subject  by  pressing  down  on  a  pressure  sensor 
mounted  on  the  manipulandum  with  a  finger  from  the  subject’s  intact  hand  (sessions  4-8,  post-implantation 
days  14-26). 

2.6.2.  Simultaneous  multielectrode  stimulation  (Subject  1-M).  Stimulation  was  delivered  via  multiple 
electrodes  simultaneously  to  investigate  if  the  subject  could  discern  multiple  electrically  evoked  percepts 
simultaneously.  Stimulation  sessions  were  performed  where  the  subject  was  cued  that  a  trial  began,  but  did 
not  know  whether  stimulation  was  delivered  via  either  of  two  electrodes,  both  electrodes  simultaneously, 
or  no-stimulation  was  delivered.  Such  blinded-trial  data  was  collected  during  three  different  experimental 
sessions  on  post-implantation  days  13,  19,  and  26.  Electrodes  with  different  inter-electrode  distances  were 
chosen,  and  supra-threshold  stimulation  amplitudes  were  used  (day  13  -  electrodes  16  and  19,  1200  pm 
distance,  1 9  pA  and  25  pA  thresholds;  day  1 9  -  electrodes  1 9  and  20,  400  pm  distance,  47  pA  and  1 8  pA 
thresholds;  day  26  -  electrodes  20  and  46,  1442  pm  distance,  18  pA  and  10  pA  thresholds).  For  each  trial, 
the  subject  had  to  record  the  size,  location  and  intensity  of  the  evoked  percept  on  anterior  and/or  posterior 
maps  of  a  hand.  If  no  percept  was  evoked,  the  subject  reported  no  sensation. 

2. 6. 3.  Stimulation  thresholds  for  all  96  electrodes  over  30  days  (Subject  2-  U).  All  96  U  SEA  microelectrodes 
were  individually  stimulated  (six  sessions  at  200  Hz  on  post-implantation  days  6-25;  two  sessions  at  20  Hz 
post-implantation  days  26-27)  and  the  threshold,  location,  size,  quality  and  intensity  of  each  evoked  percept 
was  mapped  using  custom  built  software  (LabVIEW  software,  National  Instruments  Corp.,  Austin,  TX, 
USA).  Subject  2-U  controlled  when  stimulation  was  delivered,  but  the  experimenter  controlled  the 
amplitude  (1-100  pA  in  steps  of  1-5  pA)  and  pulse  train  duration  (0.2  or  2  s).  The  quality  of  the  percept 
was  designated  from  a  list  (tingle,  pressure,  vibration,  hot,  cold)  or  could  be  defined  using  their  own  words. 
When  a  percept  was  faint  or  the  subject  was  unsure  of  the  sensation,  stimulation  could  be  repeated  until 
they  were  sure  the  percept  was  electrically  evoked  (as  opposed  to  a  phantom  limb  sensation). 

2.6.4.  Stability  of  percept  location  over  30  days  (Subject  2-U).  All  of  the  electrodes  that  evoked  a  percept 
during  each  stimulation  session  when  each  electrode  in  the  entire  array  was  stimulated  at  200  Hz  were 
analysed  (post-implantation  days  10-25;  Subject  2-U).  A  percept  was  considered  stable  if  it  remained  in  a 
localized  anatomical  region  of  the  phantom  hand,  defined  as  one  of  the  following:  anterior  or  posterior  of 
a  particular  finger,  the  palm,  or  the  back  of  the  hand.  If  a  portion  of  an  evoked  percept  was  located  on  the 
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border  separating  two  anatomical  locations  (e.g.,  over  the  metacarpophalangeal  joint),  the  percept  was 
considered  within  the  location  where  the  majority  of  the  sensation  occurred;  however,  if  the  size  of  the 
percept  spread  beyond  the  border  then  the  percept  was  not  considered  stable.  The  first  stimulation  session 
(post-implantation  day  6;  Subject  2-U)  resulted  in  an  incomplete  mapping  of  the  array  where  maximum 
stimulation  was  not  delivered  to  all  96  electrodes.  The  last  two  sessions  (post-implantation  days  26  and  27; 
Subject  2-U)  were  mapped  using  a  different  frequency  (20  Hz).  These  three  stimulation  sessions  were  not 
included  in  the  spatial  stability  analysis. 

3.  Results 

The  results  presented  herein  demonstrate  that,  using  a  100  microelectrode  array  implanted  into  human 
peripheral  nerves,  up  to  13  different  movements,  equivalent  to  8  DOF,  could  be  decoded  and  a  maximum 
of  86  percepts  could  be  evoked. 

3.1.  Electrode  impedances 

For  Subject  1-M,  the  mean  in  vivo  impedance  for  working  electrodes  over  the  29-day  implant  was  222  ± 
133  kQ  (mean  ±  std,  n  =  7  sessions;  Figure  1(a)).  Due  to  mechanical  failure  of  the  lead  wires,  the  number 
of  working  electrodes  dropped  from  93  to  20  by  the  end  of  week  2,  and  then  to  4  by  the  end  of  week  4.  In 
Subject  2-U,  the  percutaneous  connector  was  sutured  to  the  skin  to  minimize  strain  applied  to  the  wires, 
and  no  systematic  failures  indicating  wire  breakages  occurred.  The  mean  impedances  for  the  working 
electrodes  over  the  31-day  implant  for  this  subject  was  143  ±  76  kQ  (mean  ±  std,  n=13  sessions;  Figure 
1(b)),  and  the  number  of  working  electrodes  was  87  ±  5  (mean  ±  std).  Impedances  were  found  to  vary 
significantly  over  the  course  of  the  study  (p  <  0.0001,  Kruskal- Wallis  test,  Subject  2-U),  with  an  increase 
from  post-implantation  days  3  to  10. 

3.2.  Recording  and  decoding  of  motor  intent  with  high-count  electrode  arrays 

Neural  recordings  were  made  while  the  subjects  were  asked  to  make  volitional  flexion,  extension  or 
abduction  movements  of  their  phantom  fingers  throughout  the  study  period.  Examples  of  action  potentials 
recorded  from  the  same  electrodes  over  time  are  shown  in  Figure  l(c-d)  for  each  subject.  In  Subject  1-M, 
an  average  of  7  ±  9  electrodes  recorded  action  potentials  for  the  6  recording  sessions  with  a  maximum  of 
20  electrodes  recording  action  potentials  on  post-implantation  day  3  (Figure  1(e)).  In  Subject  2-U,  an 
average  of  22  ±  7  electrodes  recorded  action  potentials  for  the  1 3  recording  sessions  with  a  maximum  of 
40  electrodes  recording  action  potentials  on  post-implantation  day  7  (Figure  1(f)).  The  mean  SNR  of  the 
action  potentials  recorded  during  all  sessions  for  each  subject  were  5.2  ±  2.3  for  Subject  1-M  and  5. 1  ±  2.8 
for  Subject  2-U  (Figure  l(e-f)).  For  Subject  2-U,  after  post-implantation  day  10,  there  was  no  significant 
change  in  the  SNR  of  the  action  potentials  for  the  remainder  of  the  study  (p  =  0.24,  Kruskal-Wallis  test). 

3.2.1.  Recording  neural  activity  during  different  intended  phantom  finger  movements.  Subjects  were  cued 
to  make  different  finger  movements  either  by  a  computer  cursor  or  by  movement  of  virtual  robotic  fingers. 
The  intent  to  flex  or  extend  individual  phantom  digits  produced  spatiotemporal  neural  firing  patterns  that 
were  visible  in  the  high-pass  filtered  recording  data  (Figure  2(a-b)).  The  patterns  of  neural  activity  varied 
across  electrodes  and  movement  types.  Some  electrodes  recorded  action  potentials  that  were  correlated  with 
a  single  movement  type  (Figure  2(a),  middle  finger  flexion  (MF),  electrode  44,  Subject  1-M)  or  movement 
of  a  single  digit  (Figure  2(b),  thumb,  electrode  88,  Subject  2-U);  however,  other  electrodes  recorded  action 
potentials  that  correlated  with  multiple  different  fingers  and  movement  types  (Figure  2(b),  electrodes  13  & 
44). 
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3.2.2.  Multiple  movements  could  be  decoded  offline.  Predicted  finger  positions  showed  high  correlations 
(r=0.9)  with  the  instruction  variables  for  the  best  two  movements  (IF  and  MF)  achieved  in  Subject  1-M  and 
the  best  four  movements  (TF,  IAb,  IE,  TE)  achieved  in  Subject  2-U  (Figure  2(c-d)).  Prediction  accuracy 
decreased  as  the  number  of  predicted  movements  or  DOF  increased,  with  a  value  of  r=0.48  at  13  different 
finger  movements  for  Subject  2-U  (Figure  2(f)).  To  validate  the  decode  results,  the  electrode  order  was 
randomly  shuffled  between  the  training  and  testing  sets,  and  the  firing  rate  data  was  decoded  again.  Using 
this  shuffled  data,  the  prediction  accuracy  dropped  to  r=0. 14  for  13  different  movements  (Subject  2-U).  For 
Subject  2-U,  the  median  decode  performance  for  seven  sessions  spanning  23  days  for  the  same  eight 
movements  was  r=0.62  ±  0.07  (the  5  finger  extension  movements  were  added  on  post-implantation  day  13 
and  were  not  included  in  this  analysis). 

3.2.3.  Multiple  movements  could  be  decoded  online.  Data  recorded  from  the  median  nerve  was  decoded 
online  for  middle  and  index  phantom  finger  flexion,  and  the  subject  was  able  to  move  the  cursor  to  targets 
specific  to  each  finger  individually  with  a  median  time  to  trial  completion  of  9  ±  6  s  (16  trials,  5  electrodes; 
Figure  2  (g))  (see  Supplemental  Video  1).  Data  recorded  from  the  ulnar  nerve  was  decoded  online  for  thumb 
and  little  phantom  finger  flexions,  and  the  subject  was  able  to  independently  and  proportionally  control 
each  of  the  virtual  robotic  fingers  with  a  median  time  to  trial  completion  of  2  ±  5  s  (79  trials,  4  electrodes; 
Figure  2(h))  (Supplemental  Video  2).  For  both  subjects,  online  decodes  were  successful  in  obtaining  2  DOF 
control.  Flowever,  some  crosstalk  existed  between  these  DOF,  making  independent  control  more  difficult. 

3.3.  Intrafascicular  microstimulation  evoked  sensory  percepts 

In  Subject  1-M,  single  electrode  stimulation  on  17  electrodes  over  the  course  of  the  study  evoked  sensory 
percepts  with  an  average  threshold  of  27  ±  20  pA  (mean±  std)  (Figure  3(a))  that  were  approximately  located 
in  an  expected  median  nerve  distribution  (Figure  3(c)).  In  Subject  2-U,  when  all  96  electrodes  were 
stimulated  individually,  an  average  of  81  electrodes  evoked  sensory  percepts  with  a  mean  threshold  of  12 
±11  pA  (mean  ±  std)  (Figure  3(b))  that  were  approximately  located  in  an  expected  ulnar  nerve  distribution 
(Figure  3(d)).  For  this  subject,  no  significant  changes  in  the  thresholds  occurred  after  post-implantation  day 
10  (p=0.46,  Kruskal-Wallis  test).  Further,  a  comparison  of  all  96  electrodes  demonstrated  that  the  mean 
percept  threshold  for  an  electrode  over  the  duration  of  the  study  (8  stimulation  sessions)  was  negatively 
correlated  (r=-0.46,  p<0.0001)  with  the  number  of  days  that  an  electrode  recorded  an  action  potential  (13 
recording  sessions)  (Figure  4(a-b)). 

3.3.1.  Detection  of  multiple  percepts  simultaneously  (Subject  1-M).  For  Subject  1-M,  blind  trial  data  was 
collected  during  three  different  experimental  sessions  on  post-implantation  days  13,  19,  and  26  that 
included  stimulation  delivered  via  one  electrode,  a  different  electrode,  both  electrodes,  or  no  electrodes. 
The  following  electrode-pair  combinations  were  chosen  to  include  varying  inter-electrode  distances  using 
supra-threshold  stimulation  amplitudes:  day  13  -  electrodes  16  (19  pA)  and  19  (25  pA)  with  a  1200  pm 
distance;  day  19  -  electrodes  19  (47  pA)  and  20  (18  pA)  at  400  pm  distance;  day  26  -  electrodes  20  (18pA) 
and  46  (10  pA)  at  1442  pm  distance.  Subject  1-M  correctly  discriminated  between  any-stimulation 
(stimulation  on  either  electrode  individually  or  both  electrodes  simultaneously)  versus  no-stimulation  with 
98%  accuracy  (58/59  total  trials;  day  13  -  19/19  trials;  day  19  -  20/20  trails;  day  26  -  19/20  trials).  Subject 
1  -M  correctly  discriminated  between  spatially  distinct  percepts  evoked  by  microstimulation  delivered  via 
one  of  two  electrodes  (n  =  5  for  each  electrode  per  session)  with  87%  accuracy  (26/30  total  trials;  day  13  - 
9/10  trials;  day  19  -  8/10  trials;  day  26  -  9/10  trials).  Spatial  discrimination  was  accurately  reported  with 
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electrodes  separated  by  400  pm  (day  19)  (Figure  3(e)).  Subject  1-M  was  also  able  to  discriminate  between 
simple  patterns  of  stimulation,  i.e.,  stimulation  via  either  electrode  individually  versus  simultaneous 
stimulation  via  both  electrodes,  with  84%  accuracy  (38/45  total  trials;  day  13-1 1/14  trials;  day  19-13/15 
trials;  and  day  26  -  14/16  trials). 

3.3.2.  Frequency  and  duration  of  stimulation  modulated  the  quality  of percepts  (Subject  1-M).  Subject  1- 
M  was  allowed  to  self-modulate  the  stimulation  frequency  from  1-100  Hz  by  pushing  down  on  the 
manipulandum  pressure  sensor  with  the  subject’s  intact  hand  with  stimulation  being  delivered  to  a  single 
electrode  at  30  pA  (post-implantation  day  14).  High  frequencies  (100-320  Hz)  evoked  an  ‘electric  shock’ 
like  quality,  and  lower  frequencies  (1-25  Hz)  and  longer  stimulation  times  (up  to  60  s)  could  evoke  more 
physiological  percepts  (e.g.,  pressure). 

3.3.3.  Location  stability  of  percepts  for  all  96  electrodes  over  time  (Subject  2-U).  Five  of  the  total  eight 
stimulation  sessions  (post-implantation  days  10-25)  resulted  in  a  complete  mapping  of  all  96  electrodes  at 
fixed  parameters  of  200  Hz,  0.2  ms  durations,  200  ps  pulse  widths,  and  amplitudes  that  varied  from  1-100 
pA.  Figure  3(f)  shows  examples  of  percepts  evoked  by  two  different  electrodes  over  these  five  sessions. 
One  of  the  electrodes  (electrode  63)  evoked  a  stable  percept  across  all  five  sessions.  The  other  electrode 
(electrode  88)  was  stable  for  two  consecutive  sessions.  A  total  of  61  electrodes  evoked  percepts  across  all 
five  sessions.  Out  of  these  6 1  electrodes,  1 8  electrodes  produced  percepts  that  were  considered  stable  within 
a  defined  region  (see  Methods  2.6)  of  the  phantom  hand  for  two  consecutive  sessions  (separated  by  1-3 
days).  A  total  of  eight  electrodes  evoked  stable  percepts  for  three  consecutive  sessions,  four  electrodes 
evoked  stable  percepts  for  four  consecutive  sessions,  and  three  electrodes  evoked  stable  percepts  across  all 
five  stimulation  sessions. 

3.3.4.  Number  and  quality  of percepts  for  all  96  electrodes  (Subject  2-U).  The  quality  of  evoked  percepts 
in  Subject  2-U  combined  across  all  five  complete  mapping  sessions  at  200  Hz  included:  76  ‘tingle’  percepts; 
7  ‘pressure’  percepts;  and  216  ‘vibration’  percepts.  For  the  last  two  stimulation  sessions,  separated  by  one 
day,  where  the  array  was  mapped  at  20  Hz,  a  total  of  76  electrodes  evoked  percepts  on  both  days,  with  17 
electrodes  evoking  stable  percepts.  The  percept  quality  evoked  during  these  sessions  included:  21  ‘tingle’ 
percepts;  19  ‘pressure’  or  ‘hair  brush’  percepts;  17  ‘vibration’  percepts;  and  96  ‘cold’  or  ‘air  brush’ 
percepts.  The  subject  noted  that  the  ‘tingle’  and  ‘vibration’  percepts  evoked  during  each  session  were  of  a 
‘painful’  quality. 

3.4.  Phantom  limb  sensations  occurred  post-stimulation  in  both  subjects 

Both  subjects  differentiated  between  phantom  limb  sensations  (the  normal  feelings  from  their  phantom 
hand)  and  phantom  limb  pains  (percepts  that  were  considered  uncomfortable).  Subjects  reported  an  increase 
in  the  occurrence  of  phantom  limb  sensations  that  took  on  the  characteristics  of  the  sensory  percepts  evoked 
by  electrical  stimulation.  At  times,  the  post-stimulation  sensations  included  percepts  such  as  ‘pressure’  or 
‘hair  brushing  on  the  skin.’  At  other  times,  the  subjects  reported  an  increased  occurrence  of  post-stimulation 
evoked  phantom  limb  sensations  that  were  painful,  and  the  quality  of  such  percepts  included:  ‘electric 
shock,’  ‘stinging,’  or  ‘tingling.’  For  each  subject,  percepts  (duration  ~  1-2  s)  began  after  the  first  day  of 
stimulation,  and  had  peak  occurrences  of  10/hr  for  subject  1-M  and  2-9/hr  for  Subject  2-U.  By  30  days  post¬ 
explantation  of  the  electrode  array,  both  subjects  no  longer  reported  phantom  sensations  that  were  of  the 
quality  of  stimulation-evoked  percepts. 
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4.  Discussion 

In  2003,  a  human  volunteer  had  a  Utah  microelectrode  array  (with  all  electrodes  of  the  same  length) 
implanted  in  his  median  nerve  for  96  days.  Although  this  study  provided  new  data  for  the  field  of  peripheral 
nerve  interfaces,  it  also  had  several  limitations  including  the  following:  stimulation  and  recording  were 
carried  out  through  20  electrodes  (out  of  the  100  electrodes  that  were  implanted);  no  action  potential  data 
were  presented  to  support  that  the  devices  were  implanted  intraneurally;  no  longitudinal  data  was  presented 
to  show  the  stability  of  the  device  capabilities  (stimulating  percepts  and  recording  neural  activity);  and 
finally,  decoding  multiple  DOF  was  not  investigated  [30,  45].  Here,  we  extended  that  work  by  investigating 
multiple  aspects  important  for  developing  future  bidirectional  neural  prostheses  based  on  high-count 
microelectrode  arrays,  including:  the  quality  of  percepts  evoked  by  microelectrode  stimulation,  frequency 
modulation  of  percept  quality,  the  number  of  different  movements  that  can  be  decoded  using  action 
potential  data,  and  the  overall  recording  and  stimulation  capabilities  of  microelectrode  array  devices  over  a 
30  day  period. 

4.1.  Stability  of  USEAs  implanted  for  30  days  and  potential  for  longer  duration  implants 
In  Subject  1-M,  the  impedances  for  the  majority  of  the  electrodes  went  out  of  specification  at  the  end  of  the 
first  week  due  to  lead  wire  breakage;  however,  four  electrodes  remained  viable  for  the  duration  of  the 
implant.  Suturing  the  percutaneous  connector  to  the  skin  appeared  to  have  reduced  the  chance  of  wire 
breakage  in  Subject  2-U  (no  observed  wire  breakages).  The  SNR  was  stable  after  day  ten  of  implantation 
for  Subject  2-U,  and  we  hypothesize  this  may  be  due  to  the  stabilization  in  the  surrounding  tissue  as  the 
acute  inflammatory  response  begins  to  resolve  [46].  Impedances  varied  significantly  over  the  duration  of 
the  study  with  an  increase  during  post-implantation  days  3  to  10.  This  variability  may  reflect  either  changes 
in  the  electrode  itself  (surface  chemistry  of  the  iridium  oxide)  or  changes  in  the  electrode-tissue  interface 
(cellular  milieu).  Furthermore,  impedances  may  stabilize  after  30  days  as  shown  in  previous  studies  of 
USEAs  implanted  in  the  feline  sciatic  nerve  [47]. 

A  robust  percutaneous  connector  or  a  telemetry  system  will  be  needed  before  applications  using 
microelectrode  arrays  achieve  clinical  utility.  Also,  the  generation  of  microelectrode  arrays  used  in  this 
study  have  maximum  electrode  shank  lengths  of  1.5  mm,  which  limits  the  cross-sectional  access  to  the 
deeper  regions  of  the  relatively  large  human  peripheral  nerves  that  are  greater  than  3  mm  in  diameter. 
Longer  electrodes  and/or  multiple  arrays  may  be  needed  to  expand  electrophysiological  access  across  the 
entire  diameter  of  the  nerves. 

A  more  robust  containment  system,  an  anchoring  system,  or  a  more  compliant  microelectrode  array  design 
may  be  required  to  achieve  the  very  long  functional  lifetimes  necessary  for  clinical  applications.  The  tissue 
response  to  indwelling  intrafascicular  electrodes  has  shown  that,  although  implantation  of  intraneural 
electrodes  results  in  tissue  damage,  viable  neurons  are  found  within  distances  (<150  pm)  needed  for  safe 
stimulation  of  and  selective  recording  from  neurons  after  >30  days  of  implantation  [47,  48,  49]. 
Additionally,  Utah  microelectrode  arrays  have  been  implanted  for  >5  year  durations  in  motor  cortex  and 
continue  to  record  neural  signals  that  can  be  used  to  control  external  devices,  suggesting  the  viability  of 
this  type  of  interface  for  longer  duration  implant  times  [10].  Both  subjects  were  given  dexamethasone  and 
minocycline  doses  in  order  to  potentially  increase  the  quality  of  action  potential  recordings  over  time  [35, 
36].  However,  additional  control  studies  are  needed  to  investigate  whether  administration  of  these  drugs 
can  improve  neural  signal  longevity  over  long-term  electrode  array  implantations. 
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In  this  study,  action  potentials  were  recorded  across  multiple  microelectrodes  for  the  duration  of  the  30  day 
implant,  which  validates  that  the  devices  remained  intrafascicularly  implanted  for  the  duration  of  the  study. 
Moreover,  electrodes  that  reliably  recorded  action  potentials  throughout  the  study  generally  evoked 
percepts  at  lower  stimulation  amplitudes,  suggesting  that  these  electrodes  were  intrafascicularly  implanted, 
while  other  electrodes  that  did  not  record  action  potentials  may  have  been  implanted  between  fascicles. 

4.2.  Neural  recordings  and  decoding  independent  and  proportional  phantom  finger  movements 

An  important  aspect  of  any  decoded  signal  for  controlling  a  prosthetic  hand  is  that  it  can  mediate 
proportional  control.  This  was  investigated  in  two  ways.  First,  we  used  a  linear  Kalman  Filter  to  decode  the 
neural  data,  which  provided  a  continuous  estimate  of  finger  position  that  was  proportional  to  some  linear 
combination  of  the  neural  signals.  Second,  we  placed  targets  at  various  positions  located  between  full 
extension  and  full  flexion.  This  required  that  the  subjects  proportionally  modulate  the  neural  signals  in 
order  to  acquire  the  targets  and  complete  the  task.  The  advantage  of  using  a  virtual  prosthetic  hand  as 
opposed  to  an  actual  robotic  hand  was  programmatic  control  of  target  size  and  distance.  With  a  virtual  hand, 
we  were  better  able  to  quantify  the  accuracy  and  timing  of  neurally-controlled  movements.  For  both 
subjects,  the  decoded  neural  activity  patterns  provided  proportional  control  of  finger  position  and, 
importantly,  such  control  was  achieved  during  the  first  recording  session  with  each  subject.  Some  crosstalk 
existed  between  the  DOF,  which  sometimes  made  it  difficult  for  the  subjects  to  independently  control 
different  digits  during  the  online  experimentation. 

The  relationship  between  the  central  representation  of  motor  control  and  the  kinematics  of  movement  is 
complex,  with  activity  in  single  neural  units  correlated  with  the  movements  of  multiple  finger  muscles  [50, 
51].  The  central  and  peripheral  nervous  systems  control  the  synergistic  biomechanics  of  the  human  hand 
[52];  however,  current  high-DOF  prosthetic  hands  do  not  implement  mechanical  synergies  [24,  25],  These 
synergies  require  co-activation  of  multiple  muscles  innervated  by  multiple  nerves.  The  relationship  between 
neural  encoding  and  the  hand  biomechanics  is  further  complicated  by  the  possibility  of  post-amputation 
neural  plasticity  and  the  specific  location  of  the  array  implantation  along  the  proximal-distal  nerve  axis. 
Future  work  is  needed  to  develop  efficient  mapping  of  these  synergistic  neural  signals  onto  the  non- 
synergistic  mechanics  of  high-DOF  prosthetic  hands,  or  to  develop  prosthetic  hands  that  accurately  model 
the  biomechanics  of  the  human  hand.  Additional  work  will  also  be  required  to  develop  new  algorithms  or 
strategies  that  optimally  decode  the  spatial-temporal  neural  activity  patterns  recorded  with  intrafascicularly 
implanted  electrode  arrays  for  independent  and  noise-free  control  of  each  DOF. 

4. 3.  Stimulation-evoked  sensory  percepts 

We  explored  the  ability  to  evoke  sensory  percepts  by  injecting  currents  into  the  peripheral  nerves  via  many 
of  the  electrodes  in  the  implanted  arrays.  In  both  subjects  spatially  discrete  somatosensory  percepts  were 
evoked  using  low  levels  of  current.  These  spatially  discrete  percepts  could  be  used  for  registering  contact 
of  the  fingers  in  a  prosthetic  hand  with  an  object  with  high  spatial  fidelity.  Translations  of  non-physiological 
percepts  into  more  physiological  percepts  was  achieved  by  varying  the  frequency  and  the  duration  of 
microstimulation,  i.e.,  the  percept  changed  from  an  ‘electric  shock-like  tingle’  to  pressure,  indicating  that 
modulation  of  microstimulation  parameters  may  be  used  to  address  sub-modalities  of  somatosensation. 
Subject  1-M  was  able  to  detect  and  discriminate  simple  patterns  of  microstimulation,  suggesting  the  more 
complex  spatio-temporal  patterns  of  stimulation  could  provide  more  complex  sensation  such  as  brushing 
or  sliding  across  the  skin.  Subject  1-M  noted: 
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(1)  “As  I’m  pressing  that  down  there  [on  the  manipulandum  pressure  sensor]  on  that 
intensity  and  moving  the  finger  a  little  bit... this  [the  sensory  percept]  stayed  on  that 
finger  as  I  was  moving  it.” 

(2)  “As  they  speed  up  [the  stimulation  frequency  increasing  from  1  to  1 00  Hz]  I  can  feel 
more  of  the  finger.  ...  It  applies  pressure  on  the  index  and  this  finger  [of  the  phantom 
hand,  indicated  by  pointing  to  the  tip  of  the  ring  finger  on  the  subject’s  intact  hand].” 

(3)  Question  from  the  experimenter:  “Could  you  use  this  stimulation  to  recreate  touch?” 

Answer  from  Subject  1-M:  “Definitely. .  .The  more  you  press  it  [subject  pressed  on 
manipulandum  pressure  sensor  changing  the  frequency  from  1  to  100  Hz]  you  can 
sense  it  [the  phantom  finger]  more.  . . .  And  you’ll  get  the  sense  of  touch,  ‘cause  that’s 
what  you  did  for  me.” 

This  subject  was  also  very  accurate  at  reporting  the  absence  of  any  sensation  when  no  microstimulation 
was  provided.  The  stability  of  the  stimulation-evoked  percepts  was  assessed,  and  indicated  that  while 
evoked  percepts  were  grossly  stable  they  did  change  over  time.  The  changes  in  evoked  percepts  likely 
results  from  micro-motion  of  the  electrode  array  relative  to  the  nerve,  and  perceptual  stability  could  be 
increased  by  improving  the  systems  used  to  contain  the  array  and  anchor  it  to  the  nerve.  The  results 
presented  here  indicated  that  implantations  of  microelectrode  arrays  into  the  peripheral  nerves  of  amputees 
could  provide  sensory  feedback  that  would  improve  the  manipulation  of  obj  ects  using  highly  dexterous 
prosthetic  hands. 

Future  studies  are  warranted  to  investigate  the  stimulation  parameters  that  evoke  other  sensations,  such  as 
proprioception,  as  well  as,  long-duration  stimulation  to  evoke  long-lasting  sensations.  Recent  studies  have 
shown  modulation  of  stimulation  intensity  with  a  time -variant  pulse  width  results  in  more  natural  evoked 
perceptions  [22]  and  employing  these  methods  may  improve  the  quality  of  percepts  produced  by 
microelectrodes.  Moreover,  Tan  et  al.  have  also  demonstrated  stability  of  percept  location  may  stabilize 
after  27  weeks  post  implantation[33],  and  thus,  longer  duration  USEA  studies  are  warranted  to  assess  such 
time  periods. 

4.4.  Post-stimulation  evoked  sensory  percepts 

The  subjects  experienced  post-stimulation  percepts  that  occasionally  had  the  uncomfortable  or  painful 
qualities  of  the  percepts  evoked  by  some  electrical  stimulation.  Importantly,  neither  of  the  subjects 
experienced  these  qualities  of  percepts  prior  to  participating  in  the  study.  Future  work  should  address 
optimizing  microstimulation  parameters  to  produce  only  physiologically  relevant  sensory  percepts,  and 
investigate  plasticity  in  the  central  motor  and  sensory  neural  representations  resulting  from  long-term 
microstimulation  of  the  peripheral  nerves  [53,  54,  55,  56,  57].  Subject  2-U  noted  the  feeling  of  the  non- 
painful  post-stimulation  percepts  in  the  subject’s  phantom  limb  sensation  diary: 

“I’ve  had  the  kind  of  fluttering  or  breath-on-the-skin  or  hair-pressure  that  I  had  often  in  the 
lab  session  today.  (Located  in  the  web  area  between  my  PIF  [phantom  index  finger]  and 
PMF  [phantom  middle  finger].)  It’s  as  if  now  that  the  sensation  has  been  awakened,  it 
keeps  registering,  regardless  of  context  -  a  little  like  a  kid  using  a  new  vocabulary  word 
liberally  or  a  cook  using  a  favourite  spice  or  herb  in  all  kinds  of  meals. . .” 
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These  “awakened”  phantom  limb  sensations  may  have  implications  for  embodiment  of  neural  prostheses. 
Consistent  stimulation  of  the  nerve  may  engage  or  reactivate  central  neural  circuits  that  encode  the 
representation  of  the  phantom  or  prosthetic  limb.  This  embodiment  of  a  prosthetic  limb  could  be  enhanced 
by  coupling  the  nerve  stimulation  with  simultaneous  input  from  other  sensory  modalities,  e.g.  vision  of  the 
limb  being  touch. 

5.  Conclusion 

Numerous  electrodes  on  the  arrays  recorded  neural  activity  patterns  from  residual  nerves  that  were 
volitionally  generated  by  the  intention  to  flex,  extend  or  abduct  individual  digits  in  the  subjects’  missing 
hand.  Up  to  thirteen  movement  types  could  be  decoded  offline  (r=0.48),  and  proportional  control  of  multiple 
digits  on  a  virtual  prosthetic  hand  was  achieved.  Stimulation  of  up  to  96  electrodes,  either  one-at-a-time  or 
via  small  groups  simultaneously,  evoked  multiple  percepts  that  were  spatially  distributed  across  the 
phantom  hands  in  anatomically  appropriate  distributions.  The  relatively  large  number  of  channels  of  motor 
and  sensory  information  provided  by  the  microelectrode  arrays  indicate  that  such  arrays  can  serve  as  a 
neural  interface  for  controlling  high-DOF  prosthetic  limbs.  The  subtle  verbal  descriptions  of  evoked 
perceptions  from  the  subjects  indicated  an  attempt  to  integrate  the  sensations,  either  cognitively  or  perhaps 
due  to  neural  plasticity,  into  their  subjective  bodily  representation.  Patients  outfitted  with  a  highly  dexterous 
prosthetic  limb  controlled  through  such  a  bi-directional  peripheral  nerve  interface  might  begin  to  think  of 
the  prosthesis  not  as  a  piece  of  hardware  attached  to  their  arm,  but  rather,  as  an  integral  extension  of 
themselves. 
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Figure  1.  Electrode  impedances  and  action  potential  recording  quality.  The  left  column  of  panels  (a),  (c), 
and  (e)  show  data  from  Subject  1-M;  and  the  right  column  of  panels  (b),  (d),  and  (f)  show  data  from  Subject 
2-U.  (a-b)  Box-and-whisker  plots  of  impedances  taken  pre-implantation  in  saline  and  post-implantation 
throughout  the  study.  The  number  below  each  boxplot  is  the  number  of  working  electrodes,  (c-d)  Action 
potentials  recorded  on  example  electrodes  at  two  time  points,  one  early  and  one  late  in  the  study,  from  each 
subject,  (e-f)  Box-and-whisker  plots  of  signal-to-noise  for  all  action  potentials  recorded  on  all  electrodes 
throughout  the  study.  The  number  below  each  boxplot  is  the  number  of  electrodes  that  recorded  action 
potentials  during  the  individual  experimental  session. 
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Figure  2.  Decoding  volitional  phantom  finger  movements  from  peripheral  nerve  action  potentials.  The  left 
column  of  panels  (a),  (c),  (e),  and  (g)  show  data  from  Subject  1-M;  and  the  right  column  of  panels  (b),  (d), 
(f),  and  (h)  show  data  from  Subject  2-U.  (a-b)  High-pass  filtered  neural  recordings  (solid  blue)  made  by  three 
electrodes  during  phantom  finger  movements  for  Subject  1-M  (post-implantation  day  3)  and  Subject  2-U 
(post-implantation  day  24).  Superimposed  is  the  instructed  finger  movement  (dashed  black).  Action 
potentials  can  be  seen  in  the  high-pass  filtered  data  extending  out  of  the  noise  during  the  instruction  period, 
(c-d)  Kalman  filter  predictions  for  the  best  two  movements  for  Subject  1-M  and  the  best  four  movements  for 
Subject  2-U.  Only  a  subset  of  trials  is  shown.  Prediction  correlations  of  0.9  were  achieved  using  multi-unit 
firing  rates  calculated  from  unsorted  spike  events  on  18  electrodes  (Subject  1-M,  post-implantation  day  3) 
and  55  electrodes  (Subject  2-U,  post-implantation  day  24).  (e-f)  Box-and-whisker  plots  of  offline  Kalman 
filter  performance  for  all  combinations  of  available  movements  using  the  same  electrodes  as  in  (c-d).  (g-h) 
Examples  of  Kalman  filter  performance  during  real-time  decode  sessions  for  each  subject.  Boxes  represent 
trial-by-trial  target  locations  and  the  traces  represent  the  Kalman  filter  predictions  for  the  intended  finger 
movement.  To  successfully  complete  a  trial,  the  predicted  finger  movement  must  enter  and  remain  in  the 
target  box  for  a  specified  hold  duration  (3000  (g)  and  300  (h)  ms).  Asterisks  highlight  verbally-stated 
volitional  movements  made  by  the  subject  in  the  absence  of  a  target. 
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Figure  3.  Electrical  stimulation  can  evoke  spatially  distinct  and  stable  sensory  percepts.  The  left  column  of 
panels  (a),  (c)  and  (e)  show  data  from  Subject  1-M;  and  the  right  panels  (b),  (d),  and  (f)  show  data  from 
Subject  2-U.  (a-b)  Sensory  percept  thresholds  over  the  duration  of  the  experimental  period  for  both  subjects. 
For  subject  1-M,  only  a  subset  of  electrodes  was  stimulated  each  session.  For  subject  2-U,  except  on  post¬ 
implantation  day  6,  all  96  electrodes  were  stimulated  each  session.  The  number  below  each  boxplot  is  the 
number  of  electrodes  that  evoked  a  sensory  percept,  (c-d)  Results  from  intrafascicular  stimulation  of  human 
median  and  ulnar  nerves  show  evoked  sensations  in  the  phantom  hand  that  approximately  followed  the 
expected  spatial  distributions  for  each  nerve.  The  cumulative  data  from  all  microstimulation  sessions  is 
shown,  (e)  Single  electrode  stimulation  (electrode  19  at  47pA;  electrode  20  at  15pA)  produced  discrete 
sensory  percepts,  and  when  the  same  electrodes  were  stimulated  simultaneously  the  two  discrete  percepts 
could  be  perceived  simultaneously  (Subject  1-M).  The  inter-electrode  distance  in  this  example  was  400  pm. 
(f)  Examples  of  percepts  that  were  evoked  by  two  different  electrodes  with  one  percept  that  was  considered 
stable  across  all  five  sessions  (electrode  63;  top  row  of  hands)  and  another  that  was  stable  for  two  consecutive 
sessions  (electrode  88;  bottom  row  of  hands).  Marked  locations  were  felt  simultaneously  (Subject  2-U). 
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Electrodes 


Figure  4.  Spatial  distributions  of  percept  thresholds  and  action  potential 
recordings  for  all  96  electrodes  over  30  days,  (a)  Mean  threshold  needed 
to  evoke  a  sensory  percept  via  stimulation  on  each  individual  electrode 
over  the  duration  of  the  study  (8  total  stimulation  sessions,  last 
stimulation  session  on  day  27  post-implantation),  (b)  Number  of  days 
that  each  electrode  recorded  at  an  action  potential  over  the  duration  of 
the  study  (13  total  recording  sessions,  last  session  on  day  27  post¬ 
implantation). 
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Subject  1 -Median  Subject  2-Ulnar 


Supplementary  Figure  1.  Upper  panels  show  photographs  of  intrafascicular  USEA  implantation  into  the 
median  (Subject  1-M)  and  ulnar  (Subject  2-U)  peripheral  nerves.  The  device-nerve  interfaces  were  contained 
within  an  organic  nerve  wrap  (clear  material  surrounding  the  implant)  and  secured  with  vascular  clips  (3-5 
clips  are  seen  in  each  photo)  to  minimize  movement  of  the  device  and  allow  containment  of  the  reference 
wires  proximal  to  the  arrays.  Lower  panels  show  the  experimental  setup  primarily  used  by  each  subject  for 
volitionally  controlling  either  a  computer  controlled  display  (Subject  1-M)  or  the  digit  of  a  virtual  hand 
(Subject  2-U).  In  the  images  shown,  the  neural  information  corresponding  to  the  phantom  movement  was 
decoded  in  real-time  and  used  to  control  the  virtual  movement. 


Page  20  of  2  3 


Supplementary  Video  1.  Real-time  decode  of  index  finger  position  in  Subject  1-M  on  post-implantation 
day  10.  The  subject  achieved  proportional  control  of  the  finger  position  on  his  first  attempt  of  controlling 
the  virtual  prosthetic  hand.  The  subject  was  able  to  position  the  finger  at  halfway  flexed  and  fully  flexed 
following  verbal  instructions. 

Supplementary  Video  2.  Real-time  decode  of  thumb  and  little  finger  positions  in  Subject  2-U  on  post¬ 
implantation  day  24.  The  subject  was  able  to  simultaneously  control  the  thumb  and  little  finger  and  move 
them  to  visual  targets  presented  at  two  different  locations. 
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